Modern online assessment systems need to generate a plurality of similar, unique questions for multiple students both to reduce the possibility of cheating when students share answers to questions and to provide a basis for student drilling or practicing.
To create the plurality of unique question variants, online assessment systems require that question authors generate a question template from which a plurality of similar, unique question variants can be produced. This question authoring effort can lead to large amounts of programming effort on the part of question authors as well as requiring substantial subject matter expertise. For problems that involve real-world information, this programming task can be even more complicated due to the need for a question template to include a wide variety of real-world data necessary to produce the plurality of unique questions. Previous solutions required a question author to create each question template from scratch. Multiple question templates could not share a common library of code, or a common library of real-world facts and information.
Previous online assessment systems that do not employ templates require a large number of very similar unique static questions, with one similar static question chosen at random to be delivered to a student taking an assessment test. In any case, a question author (who authors question templates or static questions) had to copy and paste a large amount of logic and data between similar static questions or question templates.
Previous solutions required question authors to generate several pieces of information within each question template, including some or all of the following:                The text of the question.        A list of variable pieces of information within the question.        A set of constraints on each of the variables (e.g., integer values in the range −5 to +5, or a letter from the set [‘x’,‘y’,‘z’]).        One or more equations or other means to determine the correct answer to the question for each unique question variant generated by selecting random values for the variables that meet all of the constraints.        If the question is a multiple choice question, some means of specifying possible false answers (‘distractors’) for the unique question variants.        Descriptive text or explanation text displayed when a correct answer is given.        Descriptive text or explanatory text displayed when an incorrect answer is given.        One or more hints that could be provided to a student on request, or that could be provided automatically if the student provides an incorrect answer to the question.        One or more graphics and/or images that help explain the question.        
In previous systems, all of these pieces of information were provided by question authors through some manual process. Specifically, several current products include manual methods for a question author to specify the variable information within a question template and to specify the constraints on each variable. This requires that the question author specifies these variables to the testing system, and for each variable sets one or more constraints so that each variable will only generate unique values that satisfy this constraints. This is a labor-intensive task for a question author who is creating a large number of question templates. Although some expert system or artificial intelligence (AI) techniques may be used to help instructors select questions, to help a software system automatically grade assessments (e.g., homework, quizzes, tests, exams, etc.), or to help students with expert-based dynamic tips, question/test authoring remains a mostly manual task for question authors.